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Executive  Summary 

Background 

The  challenges  and  risks  to  US  ground  forces  in  the  contemporary  operational 
environment  (COE),  particularly  with  respect  to  Irregular  Warfare,  are  well  known.  Since  the 
end  of  World  War  II,  there  has  been  a  disproportionate  burden  in  casualties  bom  by  the 
Infantry  relative  to  other  branches  of  the  US  military.  It  is  believed  that  virtual  simulation, 
with  proven  effectiveness  in  training  Airmen  and  Sailors,  can  improve  the  safety  and 
effectiveness  of  Marines  and  Soldiers  by  aiding  them  in  developing  complex  and  intuitive 
decision  skills  under  conditions  of  stress.  Additionally,  the  phenomenon  of  “strategic 
compression”,  pushing  responsibility  for  decisions  with  far-reaching  consequences  to  the 
lowest  echelons  of  military  organizations,  requires  the  means  of  developing  effective 
cognitive  decision-making  capabilities  for  US  ground  forces  small  unit  leaders. 

With  the  objective  of  studying  concepts  of  immersive  training  simulation  for  Marine  Corps 
Small  Units,  a  Naval  Research  Advisory  Committee  (NRAC)  study  was  commissioned  by 
Mr.  Sean  Stackley,  Assistant  Secretary  of  the  Navy  (Research,  Development  and  Acquisition) 
and  Lt  General  George  Flynn,  USMC  (Commander,  Marine  Corps  Combat  Development 
Command).  The  Terms  of  Reference  included  six  principal  areas  of  concentration:  (1)  the 
decomposition  of  the  small  unit  immersive  simulation  training  problem,  (2)  Identifying  the 
desired  effects  of  this  training,  (3)  examining  the  metrics  necessary  to  gauge  training 
effectiveness,  (4)  reviewing  current  and  developing  virtual  training  methods,  (5)  evaluating 
current  science  and  technology  (S&T)  initiatives  and  (6)  recommending  technology  solutions, 
investments  and  developments. 

NRAC  leadership  composed  a  panel  of  NRAC  members  including  representatives 
with  industry,  medical,  military,  academic  and  acquisition  backgrounds  including  retired 
USMC  General  Officers  and  Naval  Flag  Officers.  The  study  engaged  stakeholders  and 
participants  in  Government,  academia  and  industry. 
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Findings 

At  the  outset,  the  panel  found  that  there  is  no  broad  consensus  on  the  meaning  of 
immersion ,  with  some  in  the  “presence”  research  community  favoring  a  set  of  objective 
criteria  (spanning  the  number  of  senses  addressed  and  the  fidelity  of  sensory  impressions)  and 
others  favoring  a  definition  relying  on  a  subjective  impression  of  engagement  and  suspension 
of  disbelief.  The  panel  favored  the  later  definition  in  its  deliberations. 

The  panel  adopted  a  learning  hierarchy  framework  for  training  USMC  small  units 
spanning  (a)  declarative  knowledge  of  facts,  (b)  consolidation  and  acquisition  of  procedural 
knowledge  and  (c)  higher  order  skills  and  team  organization.  Against  this,  the  panel 
considered  a  range  of  virtual  simulation  alternatives. 

Virtual  reality,  in  which  the  user  experiences  a  synthetic  environment  exclusively, 
typically  involves  a  desktop,  large-format  or  head-mounted  display  (HMD)  interface  for 
increasing  levels  of  immersion.  Mixed  reality  includes  augmented  virtuality,  in  which  the 
user  experiences  a  synthetic  environment  with  selected  real-world  components  that  appear  in 
scene  graph  masks  (i.e.  holes  in  the  scene  image),  typically  with  a  see-through  optical  HMD. 
In  augmented  reality,  the  user’s  primary  experience  is  of  the  real  world  with  synthetic  objects 
occluding  the  real-world,  typically  with  a  see-through  optical  HMD  interface. 

This  range  of  virtual  simulation  alternatives  suggests  a  potential  fit  with  the  hierarchy 
of  training  challenges.  Thus  an  augmented  reality  capability,  with  the  ability  to  move  freely 
in  a  physical,  three-dimensional  space,  might  be  a  good  fit  for  learning  higher  order  skills  and 
organization  while  desktop  virtual  reality  might  be  suited  to  learning  declarative  knowledge 
of  facts.  Whatever  the  level  of  training,  however,  the  inherent  non-sequential  nature  of 
simulation  training  more  naturally  supports  the  development  of  higher-order  cognitive 
capabilities  than  linear  task  and  part-task  training. 

There  is,  perforce,  an  issue  of  the  maturity  of  the  enabling  technologies  for  some  of 
these  virtual  training  approaches.  Augmented  reality,  for  example,  relies  on  accurate 
position-location  information  (PLI)  along  with  practical,  see-through  optical  HMD’s.  The 
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panel  determined  that  PLI,  at  a  scale  in  footprint  and  number  of  participants  required  for 
USMC  small  unit  training,  was  not  yet  available.  Similarly,  lightweight,  inexpensive  see- 
through  optical  HMD’s  will  require  additional  S&T  investment. 

For  current  and  upcoming  virtual  training  solutions,  there  is  the  additional  challenge  of 
a  general  lack  of  metrics  for  determining  the  effectiveness  of  alternatives.  This  has 
contributed,  the  panel  believes,  to  a  lack  of  consensus  of  the  value  of  simulation  training  for 
USMC  small  units.  Notwithstanding  the  current  availability  of  several  virtual  simulation 
tools,  there  is  no  actual  guidance  on  their  employment  in  the  USMC  Infantry  Training  and 
Readiness  (T&R)  manual.  Without  metrics,  it  is  difficult  to  provide  guidance  about  the  value 
of  virtual  systems  as  an  alternative  to,  for  example,  live  training  exercises.  Moreover,  the 
Mission  Essential  Tasks  (MET’s)  in  the  T&R  manual  do  not  include  the  cognitive  aspects  of 
irregular  warfare.  Thus  there  are  currently  no  tasks  against  which  to  consider  virtual  training 
tools  that  might,  for  example,  build  meta-cognition  skills. 

The  panel  found  potential  for  addressing  the  absence  of  metrics  for  virtual  simulation 
products  in  a  study  conducted  by  Canadian  researcher,  Dr.  Paul  Roman  of  the  Royal  Military 
College  of  Canada.  In  a  study  published  in  December  2008,  Dr.  Roman  found  that  the 
cognitive  decision-making  skills  of  Canadian  tank  commander  students  increased 
dramatically  with  a  portion  of  the  conventional  classroom  curriculum  was  re-directed  to 
virtual  simulation  exercises. 

His  metrics,  however,  derived  exclusively  from  the  subjective  assessment  of  trained, 
experienced  evaluators.  The  dependent  variable  in  this  experiment  was  the  pass/fail  outcome 
for  the  students  in  the  course.  The  experimental  group  in  the  Roman  protocol  passed  the 
course  earlier  and  in  greater  numbers  than  the  control  group. 

Until  other,  possibly  computationally-based,  automated  approaches  may  be  found,  the 
panel  believes  that  the  systematic  application  of  assessment  by  trained  evaluators  considering 
the  outcome  of  training  events  will  provide  the  hard  data  necessary  for  evaluating  training 
alternatives. 
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Notwithstanding  the  limitations  of  available  technological  components  and  metrics, 
the  USMC  has  seized  the  initiative,  building  an  Infantry  simulator  at  Camp  Pendleton,  the 
Infantry  Immersive  Trainer  (IIT).  With  limitations  in  throughput  and  availability,  the  IIT  is 
intended  to  inform  a  planned  Program  of  Record,  the  Squad  Immersive  Training  Environment 
(SITE).  According  to  users,  much  good  has  already  come  of  this  pioneer  effort.  The  panel 
noted  some  immediate  enhancements  (e.g.  sound  design/reinforcement)  that  could  add  to  the 
immersive  qualities  of  the  IIT. 

In  future  implementations,  the  requirements  of  IIT-type  facilities  could  benefit  from 
cognitive  task  decomposition:  teasing  apart  the  desired  training  end-states  from  immersive 
simulation  training  events.  This  would  enable  trainers  to  maximize  the  effect  of  time  spent  in 
IIT-type  facilities  and  modulate  the  training  that  the  facilities  would  be  required  to  provide. 

Recommendations 

For  immediate  action,  the  panel  recommends  (a)  the  implementation  of  systematic, 
subjective  assessment  methods  to  evaluate  simulation  alternatives,  (b)  the  development  of 
cognitive,  irregular  warfare-related  MET’s,  (c)  various  enhancements  to  the  IIT  along  with  (d) 
cognitive  task  decomposition  to  support  an  “end-to-end”  training  solution. 

With  an  eye  to  the  future,  the  panel  considered  the  current  USMC  S&T  investment 
(“Code  30”  in  the  Office  of  Naval  Research  Budget)  in  addition  to  the  broader  Department  of 
Defense  technology  investment.  The  panel  found  that  the  current  portfolio  is  under-resourced 
in  terms  of  supporting  the  technology  components  required  for  practical  augmented  reality, 
the  likely  next  step  in  immersive  small  unit  virtual  simulation  training.  The  panel  also 
determined  that  the  SITE  program  needs  an  integration  laboratory  (most  likely  at  Camp 
Pendleton’s  IIT)  to  validate  the  development  of  technology  components  as  they  become 
available  and  the  ways  these  tools  will  be  used  in  the  SITE  program. 


viii 


With  this  approach,  the  panel  believes  Marines,  in  the  future,  will  face  their  first 
combat  in  a  simulator:  not  on  the  battlefield.  The  panel  believes  further  this  will  make  a 
great  difference  in  avoiding  casualties  and  saving  lives. 
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Core  Inquiry 


What  is  the  role  of  immersive  simulation  in 
training  and  assessing  a  USMC  squad  as  an 
effective  weapons  system? 


"We  need  a  giant  leap  forward  in  our  simulated  training  environment  for 
small  units  in  ground  combat  ...to  replicate  to  the  degree  practical  using 
modern  simulation,  combat  scenarios  that  will  test  our  small  units  ..." 

Gen  J.  M.  Mattis,  USMC 
Commander,  U.S.  Joint  Forces  Command 

"The  goal  must  be  to  take  training  capabilities  to  the  next  level  and  fuse 
current,  emerging,  and  future  live  and  virtual  technologies  to  create  a 
fully-immersive  live/virtual  training  environment" 

LtGen  G.  F.  Flynn,  USMC 

Deputy  Commandant,  Combat  Development  and  Integration 


Combat  operations  commencing  in  Afghanistan  (2001)  and  Iraq  (2003)  have  been 
marked  by  irregular  warfare  involving  Marines  and  Soldiers  fighting  insurgents 
employing  improvised  explosive  devices  (IEDs),  roadside  bombs,  handheld  antitank 
grenade-launchers  firing  RPGs  (Rocket-Propelled  Grenades),  suicide  bombers,  small 
arms  fire  and  mortars.  The  vast  majority  of  U.S.  causalities  have  been  sustained  in  small 
unit  ground  combat  operations. 


The  Panel  focused  on  the  question  of  what  role  immersive  simulations  ought  to 
play  in  training  and  assessing  Marines  prior  to  entering  the  irregular  warfare 
environment.  We  looked  at  current  and  emerging  immersive  simulation  techniques  and 
technologies  that  might  enable  young  Marine  squad  members  to  become  better  decision¬ 
makers:  better  critical  thinkers. 


As  shown  in  the  above  quotations,  senior  leadership  fully  expects  that  the 
research  efforts  supporting  the  infantry  -  both  Army  and  Marine  Corps  -  should  shape 

evolving  immersive  simulation  developments  to  address  their  complex  training 
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requirements.  Whether  it  is  the  “leap-ahead”,  that  General  Mattis  (USJFCOM)  seeks  or 
the  secular  evolution  of  fused  capabilities  that  Lieutenant  General  Flynn  describes, 
remains  to  be  seen.  In  this  study,  the  panel  investigated  ongoing  efforts  in  this  area  with 
an  eye  to  understanding  the  remaining  barriers  to  fully  immersive  small  unit  simulation 
training  for  the  COE.  With  an  understanding  of  these  remaining  challenges,  a  way  ahead 
can  be  formulated  to  realize  the  vision. 
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NRHC  Sponsor  and  Panel  Membership 

Studv  Panel  Members 

Studv  Sponsor 

•  Mr.  James  H.  Korris  -  Chair 

•  Mr.  Sean  J.  Stackley 

-  Creative  Technologies  Inc. 

-  Assistant  Secretary  of  the  Navy,  RD&A 

•  Dr.  A.  Michael  Andrews,  II  -  Co-Chair 

•  LtGen  George  J.  Flynn,  USMC 

-  L-3  Communications 

-  DC,  CD&I 

•  Dr.  Regina  E.  Dugan 

-  RedX  Defense 

•  MajGen  Paul  Fratarangelo,  USMC  (Ret.) 

Executive  Secretariat 

-  Private  Consultant 

•  Mr.  E.  Ray  Pursel  -  Executive  Secretary 

•  Dr.  Helena  S.  Wisniewski 

-  MCWL  (Experiment  Div) 

-  Corporate  Director 

•  Mr.  B.  Greg  Kesselring  -  Asst.  Executive 

•  Dr.  Anna  D.  Johnson-Winegar 

Sec. 

-  Private  Consultant 

-  MCWL(OSTI) 

•  Dr.  Jane  A.  Alexander 

-  Private  Consultant 

•  CAPT  R.  Robinson  Harris,  USN  (Ret.) 

-  Lockheed  Martin  Maritime  Systems 

•  RADM  Charles  B.  Young,  USN  (Ret.) 

-  Oceaneering  International,  Inc. 

•  Dr.  Robert  S.  Carnes,  MD 

-  Batelle  Memorial  Institute 

The  Panel  -  with  individuals  from  industry,  medical,  military,  and  acquisition 
backgrounds  -  brought  a  broad  range  of  technical  expertise  to  this  Study. 

Our  Marine  Corps  study  sponsor,  Lt  General  Flynn,  IJSMC,  ensured  Panel  access 
to  all  of  the  key  players  and  existing  systems  that  were  relevant  to  the  fact-finding  phase. 
The  sponsor-designated  Executive  Secretaries  provided  the  Panel  with  exceptional 
support  including  key  insights  into  the  state-of-the-art  training  capabilities  of  the  Marine 
Corps. 
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Fact-Finding 


Marine  Corps/Navy 

CG,  MCCDC  (Study  Sponsor) 

CG,  MCWL 

MCWL,  Modeling  and  Simulations  Branch 
Training  and  Education  Command 
Naval  Air  Warfare  Training  Center  Training  Systems 
ONR  Code  30 

USMC  Program  Manager,  Training  Systems 
Naval  Research  Lab 

Infantry  Immersion  Trainer  Brief  and  Demonstration 

Combat  Convoy  Trainer  Demonstration 

Yankee  Huey  and  Zulu  Cobra  Trainer  Demonstration 

University/National  Lab 

University  of  Central  Florida,  Media  Convergence  Lab 

and  Institute  for  Simulation  and  Training 

Royal  Military  College  of  Canada 

Sandia  National  Laboratories 

Institute  for  Creative  Technologies  Visit  and  Briefs 


Other  Services  and  Agencies 

TCM  Virtual 

PEO,  Simulation,  Training  and  Instrumentation  (STRI) 
Future  Immersive  Training  Environment  (FITE),  JCTD 
Medical  Science  Advisor  to  the  CJCS 
US  Army  Research  Institute  (Behavioral  &  Social 
Sciences) 

US  Army  Armor  Center 
Fires  Battle  Lab,  US  Army 

US  Army  Director(Research  &Laboratory)Management 
DARPA 

US  Army  RDECOM  Sim  &Training  Technology  Center 

Industry 

A-T  Solutions 
Boeing 

Forterra  Federal  Systems 
L-3  Communications 
Lockheed  Martin 
MAK  Technologies,  Inc 
MYMIC 

Soar  Technologies 

Total  Immersion  Software,  Inc.. 


The  NRAC  Study  Panel  cast  a  wide  net  in  terms  of  fact-finding.  The  Panel  made 
every  effort  to  talk  to  everyone  who  had  a  stake  in,  or  who  was  active  in  the  area  of 
immersive  simulation.  Briefings  were  received  from  a  wide  variety  of  organizations  and 
individuals  in  the  Marine  Corps  and  Army  acquisition  and  technology  offices, 
government  and  academic  researchers,  other  defense  community  members,  and 
representatives  from  small,  medium  and  large  for-profit  companies.  As,  particularly  in 
industry,  the  field  is  highly  fragmented  and  generally  populated  with  a  large  number  of 
small  businesses,  the  panel  sought  a  representative  sample  of  these  stakeholders  in 
relevant  disciplines  including  virtual  worlds,  artificial  intelligence,  virtual  simulation 
middleware,  authoring  tools  and  displays. 


The  Panel  was  surprised  by  the  lack  of  mature  information  on  research  and 
application  of  metrics  in  the  area  of  immersive  simulation  and  its  training  effectiveness. 
There  were  exceptions  -  including  areas  of  research  sponsored  by  the  Office  of  Naval 
Research  and  the  Canadian  Military.  Dr.  Fidopiastis  from  the  University  of  Central 
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Florida  had  performed  work  in  the  measurement  of  human  response  to  stressful 
conditions  in  a  simulated  environment.  Dr.  Roman  of  Canada’s  Royal  Military  College 
reported  on  the  use  of  training  metrics  to  measure  the  impact  of  simulations  for  Tank 
Commander  training.  CAPT  (sel)  Schmorrow  and  LCDR  Cohn,  previously  assigned  to 
ONR,  presented  their  work  over  a  number  of  years  that  provided  the  panel  with  a  deeper 
insight  in  the  area,  and  challenges  of  metrics. 
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Panel  Definition  of 
"Immersive  Simulation” 


•  A  simulation  that  produces  a  state  of  being 
deeply  engaged;  suspension  of  disbelief; 
involvement 

•  Immersive  Simulation  Training  Environment 

-  A  training  environment  that  includes  one  or  more 
aspects  of  simulation  (ranges  from  a  few  special 
effects  up  to  a  full  virtual  world)  that  deeply 
engages  the  trainee 


As  the  Panel  received  its  various  briefings  from  the  experts  in  the  field,  a  working 
definition  of  “immersive  simulation”  was  formed  as  shown  above.  It  is  necessarily 
subjective,  as  it  applies  to  individualized  human  stimulus  and  response.  Later  in  this 
report,  as  the  status  of  immersive  simulation  is  discussed,  we  note  that  no  single 
definition  has  emerged  to  which  all  parties  can  agree. 

It  is  interesting  to  note  that  the  application  of  immersive  simulation  to  the  training 
challenge  can  take  a  number  of  paths  -  from  a  fully  “digital”  virtual-world  simulation  to 
one  that  combines  a  real  and  virtual  environment  -  providing  the  trainee  with  a  level  of 
engagement  not  replicated  in  other  forms  of  training. 
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NRffL  Marine  Corps  Training  Principles 


•  Train  as  you  fight. 

*  Make  commanders  responsible  for  training. 

*  Use  standards-based  training. 

*  Use  performance-oriented  training. 

•  Use  mission-oriented  training. 

•  Train  the  MAGTF  to  fight  as  a  combined-arms  team. 

•  Train  to  sustain  proficiency. 

*  Train  to  challenge. 

—  MCO  1553. 3A  Unit  Training  Management 
-  NAVMC  5300.44  Infantry  T&R  Manual 


The  Panel  began  its  work  by  examining  and  embracing  the  fundamental,  foundational 
Marine  Corps  training  principles,  in  particular  the  first  one,  “Train  as  you  fight”.  Marines 
want  to  train  with  their  own  equipment  (e.g.,  rifle,  eye  wear,  helmet)  with  freedom  of 
movement  to  employ  their  combined  arms  capabilities. 
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NRRC  Immersive  Simulation  Status 

•  Lack  of  consensus  on  value  vs.  cost 

*  Lack  of  guidance  to  allow  simulation  to  accomplish 
Training  and  Readiness  Manual  syllabus  tasks 

•  Pre-deployment  Training  Plan  (PTP)  does  not 
currently  require  immersive  simulation 

*  USMC  immersive  trainers  have  limited  availability 
and  throughput 

-Typically,  one  squad  is  trained  per  evolution 
-~243  Marine  Rifle  Squads  per  division 

Conditions  are  not  set  for  full  utilization 
of  immersive  simulation 


The  Panel  learned  that  conditions  are  not  yet  set  for  full  implementation  and 
utilization  of  immersive  simulation  training  in  the  infantry  services,  including  the  USMC. 
The  Panel  noted  several  significant  barriers  or  obstacles. 


The  first  barrier  is  a  notable  lack  of  consensus  over  the  value  of  the  current 
simulation  training  methods  and  technologies.  Also,  the  very  definitions  of  “immersion” 
or  “immersive”  are  their  own  kind  of  babble.  Today,  the  terms  are  so  overused  as  to  be 
almost  meaningless.  It  is  the  adjective  du  jour  for  contractors  who  have  any  kind  of 
training  or  simulation  system  to  sell.  It  is  often  taken  to  mean  a  product  that  users  will 
find  particularly  engaging  and  compelling  -  and  whether  it  accomplishes  that  through 
objective,  measurable  means  or  through  just  the  quality  of  the  experience  -  the  net  result 
appears  to  be  the  same.  It’s  something  that  really  engages  the  user,  but  its  value  has  been 
controversial  among  USMC  leadership. 
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A  second  barrier  is  a  clear  lack  of  guidance  or  any  documented  requirement  to 
utilize  simulation  as  a  means  to  accomplish  the  tasks  in  the  USMC  Training  and 
Readiness  (T&R)  Manual.  The  latest  revision  of  the  T&R  manual  discusses  simulation 
and  encourages  its  use  as  an  effective  and  efficient  way  to  meet  the  training  requirement 
-  but  there  is  nothing  about  how  to  use  it  or  its  value  to  the  training  effort. 


This  leads  to  a  third  barrier.  The  Pre-deployment  Training  Plan  (PTP)  does  not 
require  immersive  training.  This  is  probably  due  to  the  lack  of  close-by  immersive 
simulation  facilities  for  each  MEF  -  the  fourth  significant  barrier  noted  by  the  Panel. 
With  only  one  Infantry  Immersive  Trainer  located  on  Camp  Pendleton,  it  is  neither 
convenient  nor  cost  effective  for  East  Coast  or  Hawaii-based  USMC  units.  And,  even  for 
West  Coast  Marines,  there  is  limited  throughput  available  in  the  IIT. 
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Terms  of  Reference 


Objecth/e:  Study  concepts  of  immersive  training 
simulation  to  assist  Marines  in  developing  complex  and 
intuitive  decision  skills  under  stress... 

•  Decompose  the  small  unit  immersive  simulation  training  problem 
and  identify  the  desired  effects  of  such  training 

•  Examine  the  metrics  necessary  to  gauge  training  effectiveness 

•  Identify  the  desired  effects  and  examine  the  metrics 

•  Review  current  and  developing  virtual  training  methods 

•  Evaluate  current  S&T  initiatives 

•  Recommend  technology  solutions,  investments  and  developments 


These  are  study  Terms  of  Reference  (TOR)  in  bullet  form.  The  full  text  TOR  is 
provided  as  Appendix  A. 

The  Panel’s  fact-finding  over  a  six  month  period  provided  the  background  to: 
decompose  (i.e.,  tease  apart)  elements  of  the  training  problem;  identify  the  presence  and 
suitability  (or  lack  thereof)  for  immersive  simulation  performance  metrics;  and  review 
the  existing  set  of  virtual  training  systems  as  well  as  those  in  development.  Next,  the 
Panel  reviewed  the  DOD  Science  and  Technology  investment  -  focused  primarily  on  the 
Naval  S&T  portion.  Finally,  recommendations  for  the  near  and  farther  term  were  offered. 
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Findings 
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Hierarchy  of  Training 
Obiectives/Approaches 


*  Declarative  Knowledge  of  Facts 

-  Issues:  Difficult  to  practice  skills  and  consolidate  knowledge 

<w  *  Consolidate  Declarative  and  Acquire  Procedural  Knowledge 

s;  -  Issues:  Difficult  to  acquire  higher-order  skills ,  strategic 

ro  knowledge 

o 

re  •  Higher  Order  Skills  and  Team  Coordination 

!£.  -  Issues:  Cost  of  actors;  limited  availability/throughput; 

3  support  staff;  currently  not  domain  transferable 

u  •  Higher  Order  Skills,  Team  Coordination,  and  Strategic 
5‘  Knowledge 

orq 

-  Issues:  Can  overwhelm  or  distract  early  procedural  learning; 
limited  availability/throughput;  domain  specific;  high  cost  of 
overhead 


The  Panel  felt  that  it  was  particularly  useful  to  consider  the  “training  continuum” 
in  order  to  determine  the  issues  involved  with  the  application  of  simulation  training 
products  for  cognitive  decision-making  for  Marine  small  units. 

The  straightforward  transfer  of  information  in  a  classroom  or  lecture  setting 
(“declarative  knowledge”)  for  example,  imparts  some  knowledge  of  the  presented  facts 
but  offers  little  opportunity  to  practice  skills  or  consolidate  that  which  was  learned. 
“Hands-on”  training  allows  a  single  student  to  begin  memory  consolidation  -  stabilizing 
a  “memory  trace”  for  easier  recall.  But  this  procedural  knowledge  of  a  hands-on  skill  is 
usually  limited  in  complexity  and  lacks  cross  domain,  strategic  knowledge  components. 

Collective  training  generally  requires  a  more  complex  environment  that  imparts 
higher  order  teamwork  skills.  The  current  I  MEF  Infantry  Immersive  Trainer,  is  an 
example  of  such  a  facility.  It  is,  however,  burdened  with  the  cost  of  numerous  cultural 
“role  players”,  limited  student  throughput  and  difficulty  in  changing  the  fixed  physical 
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plant  to  simulate  various  operational  environments.  The  addition  of  training  elements  that 
imbue  a  subjective  strategic  knowledge  skill  set  is  a  challenge.  This  type  of  subjective 
knowledge  permits  decision-making  under  severe  time  constraints  and  pressure.  If  not 
properly  controlled,  this  type  of  training  can  overwhelm  an  individual  at  the  outset,  and 
distract  the  reminder  of  the  training  sequence. 
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Immersive  Simulation 
Technologies 


Training  Continuum: 
Classroom 


Computer  Based  Training 


Virtual  Reality:  Desktop 
Virtual  Reality:  Projection 
Full  Immersive  Virtual  Reality 

Augmented  Reality - 

Mixed  Reality 
Full  Simulation  (does  not  exist  for  Infantry) 


Live  Training 
Combat 


There  is  a  spectrum  of  technologies  that  align  with  the  training  continuum  in  the 
previous  chart.  At  the  simplest  level,  that  of  the  classic  training  venue  of  the  classroom, 
to  arguably  the  most  immersive  -  that  of  on-the-job  training  in  live  combat  -  exist  a 
range  of  technology  applications.  The  Panel  considered  those  in  the  shaded  area  i.e., 
those  that  provide  some  level  of  reality  simulation  by  computational  means. 

The  following  descriptions  may  be  unfamiliar  to  the  reader.  Full  Immersive 
Virtual  Reality  utilizes  an  occlusional  head  mounted  display  (HMD).  The  user  is 
presented  exclusively  with  a  view  of  a  computer-generated  virtual  world  and  none  of  the 
physical  world  that  surrounds  him.  Conversely,  Augmented  Reality  is  a  simulation 
approach  where  the  user  sees  the  real  physical  world  around  him  combined  with  synthetic 
(computer  generated)  objects  inserted  into  this  “real  world”.  The  HMD,  in  this  case,  is  an 
optical  or  video  “see-through”  device  allowing  the  visual  transmission  of  the  ambient 
environment  to  the  user.  Spatial  masks  block  the  passage  of  the  ambient  setting  with 
tracked  rendered  portions  of  the  virtual  scenegraph  displayed  in  the  HMD. 
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Augmented  Virtuality  is  a  further  refinement  of  Augmented  Reality.  In  this  case, 
the  user’s  primary  experience  is  of  a  synthetic  environment  with  masks  creating  an 
interruption  in  the  scene  graph  to  allow  the  transmission  of  real  world  visual  stimuli.  In 
one  approach  to  Mixed  Reality,  synthetic  objects  are  integrated  into  a  real-world  setting 
with  discrete  displays.  Thus  a  monitor  displaying  an  avatar  in  a  physical  environment  is 
an  implementation  of  Mixed  Reality. 
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Technology  Maturity 


As  the  Panel  evaluated  critical  technologies  and  their  maturity,  two  areas  set  the 
current  limitations  that  will  ultimately  enable  practical  augmented  reality.  The  first  is  the 
area  of  display  capabilities.  The  top  left  image  is  a  current  state-of-the-art,  see-through 
optical  head  mount.  The  user  is  able  to  observe  the  real  physical  world  around  him  as 
well  as  see  computer-generated  synthetic  objects  at  the  same  time.  A  manufacturer’s 
end-state  example  is  shown  to  the  right  -  the  form  and  fit  of  a  pair  of  reading  glasses. 
The  first  typically  weighs  more  than  a  pound  while  the  glasses  are  measured  in  ounces. 
There  will  be  significant  R&D  to  achieve  the  end-state.  (Affordability  is  always  a  key 
factor  -  today  these  glasses  cost  about  $15,000  each.  The  target  price  would  be  under 
$500.) 


Similarly,  but  perhaps  less  obvious,  is  that  the  precise  location  of  every  object  in 
the  “scene”  must  be  accurately  known  in  order  to  juxtapose  and  combine  the  real  world 
with  the  synthetic  world.  Today  this  is  achievable  -  but  it’s  only  achievable  at  significant 
expense  and  in  small  interior  venues.  For  a  two,  three,  or  five-acre  outdoor  MOUT  site,  a 


26 


different  kind  of  technological  approach  is  required.  Unfortunately,  there  is  limited 
progress  in  that  direction,  and  substantial  R&D  will  once  again  be  required. 
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NRRC  Current  USMC  Simulation  Tools* 


*  Deployable  Virtual  Training  Environment  (DVTE) 

-Close  Combat:  Marines  (CCM) 

-Virtual  Battlefield  System  1  &  2  (VBS) 

-  Recognition  of  Combatants  -Improvised  Explosive 
Devices  (ROC  -  IED) 

-  Combined  Arms  Network  (CAN) 

*  Indoor  Simulated  Marksmanship  Trainer  (ISMT) 

*  Combat  Convoy  Simulator  (CCS) 

*  Infantry  Training  &  Readiness  Manual 


An  important  part  of  our  research  involved  reviewing  current  and  developing 
virtual  training  methods,  and  how  they  are  being  employed  in  the  Marine  Corps. 
Identified  in  the  USMC  Infantry  Training  and  Readiness  (T&R)  Manual  are  extant 
simulations  available  for  commanders.  These  are  primarily  desktop  computer  courses: 
the  Deployable  Virtual  Training  Environment  (DVTE);  the  Indoor  Simulated 
Marksmanship  Trainer  (ISMT);  and  the  “immersive”  mixed  reality  environment  Combat 
Convoy  Simulator  (CCS).  The  CCS  is  decidedly  “vehicle-centric”  with  some 
dismounted  training  opportunities. 


A  key  finding  is  that  the  T&R  Manual  provides  no  guidance  to  USMC 
Commanders  regarding  the  use  of  individual  training  simulations.  It  does  not  prescribe  or 
recommend  where  simulations  belong  in  the  training  syllabus,  what  other  proscribed 
training  events  they  may  replace  or  of  what  enhancement  of  training  readiness  they 
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provide  to  the  commander  who  chooses  to  employ  them  in  his  unit’s  pre-deployment 
training  syllabus. 
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Status  of  Immersive 
Simulation  Metrics 


•  Conventional  training  metrics  do  not  apply  to 
immersive  simulation. 

-  Sequential  versus  simultaneous  execution  of  tasks 

•  For  acquisitions  beyond  current  systems,  cost, 
schedule  and  performance  criteria  required 

-  Absence  of  performance  metrics  for  infantry  simulators 

-  Quantifiable  proof  of  effectiveness  desirable  for  all  training 

-  Metrics  assist  in  evaluating  reproducibility  and  retention  of 
training 

“Subjective  assessment  by  a  trained  evaluator  is  a  valid  metric.  ” 

-  Dr.  Paul  Roman,  Royal  Military  College  of  Canada 


The  Panel  was  disappointed  by  the  lack  of  metrics  for  immersive  simulation. 
Despite  significant  interest  in  many  quarters,  little  has  been  done  in  this  area.  Most 
briefers  stated  that  developing  realistic  metrics  was  just  “too  hard.”  The  operating 
assumption  is  that  because  conventional  training  is  sequential  and  immersive  simulation 
is  not,  that  conventional  metrics  do  not  apply. 

In  a  typical  task  evaluation,  a  student’s  performance  is  checked  off  as  he  proceeds 
through  a  sequential  evolution.  In  a  simulation  environment,  there  can  be  lots  of  things 
happening  at  once  and  it’s  difficult  to  measure  the  training  results  with  the  various  stimuli 
-  hence  no  metrics. 

The  current  set  of  immersive  simulation  training  systems  in  use  by  the  military 
were  acquired  without  benefit  of  metrics  used  to  support  trade-off  analyses.  Despite  the 
current  use  of  training  simulations,  there  remains  a  lack  of  performance  metrics  to  show 
their  cost/benefit.  Without  valid  metrics,  there  cannot  be  an  evaluation  of  alternative 
solutions  or  approaches. 
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Dr.  Paul  Roman  of  Canada’s  Royal  Military  College  took  a  different  approach  in 
addressing  the  need  for  metrics  in  evaluating  cognitive  decision-making  training  aids.  In 
a  series  of  experiments  to  test  the  usefulness  of  training  simulations  for  Canadian  Tank 
Officers,  he  developed  performance  metrics  based  on  the  “subjective  assessment  of 
trained  evaluators”.  Although  his  method  appears  obvious  -  not  a  particularly  innovative 
idea  -  its  application  to  a  simulation  product  is,  in  terms  of  the  panel’s  findings  with 
respect  to  ground  forces  training,  novel. 
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Expert  Evaluator 
Training  Measurement 


Control  Min  Sim  Half  Sim 

(No  Simulation)  (lday)  (2.5  weeks) 

%  Pass  on 

1st  Evaluation 

0  30%  67% 

%  Pass  by  1/2  , 

of  Evaluations 

51%  72%  100% 

%  Pass  by 

End  of  Course 

72%  83%  100% 

Source:  "Games  -  Just  How  Serious  Are  They?",  Dr.  Paul  A  Roman,  Mr.  Doug  Brown,  Interservice/Industry  Simulation  and 
Education  Conference  2008 


The  above  spreadsheet  of  Dr.  Roman’s  “%  pass”  results  clearly  shows  the  benefit 
derived  from  employing  simulation  training  (i.e.,  an  interactive,  free-play  tank 
commander  desktop  trainer)  as  part  of  the  six-week  Tank  Training  Course,  conducted  at 
the  Canadian  Combat  Training  Centre.  The  columns  represent  the  amount  of  simulated 
game  based  training  and  the  rows  the  percent  of  the  subject  groups  that  passed  the  course. 
Performance  was  based  upon  student  success  rates  as  defined  by  the  proportion  of 
students  that  passed  the  demanding  course  and  the  number  of  real-world  traces 
(evaluations  or  battle  runs)  needed  to  demonstrate  proficiency.  The  results  from  three 
consecutive  groups  taking  this  course  are  represented. 

The  first  column  represents  a  control  group  that  had  no  simulation-based  training, 
having  only  the  conventional  classroom  course  curriculum.  The  second  column 
represents  the  trainees  that  experienced  one  day  of  simulation  activity  and  5.5  weeks  of 
classroom  training.  The  third  column  group  experienced  2.5  weeks  of  simulation  training 
with  only  3  weeks  of  classroom  instruction. 
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When  the  students  completed  their  first  evaluation  or  “trace”,  the  pass  rate  for 
students  in  the  two  experimental  groups  with  a  simulation-augmented  curriculum  showed 
a  higher  pass  rate  than  the  control  group.  The  pass  rate  increased  with  a  rise  in  the 
proportion  of  simulation  time  in  the  course. 

Although  these  results  are  supportive  of  the  value  of  simulation-based  training, 
the  Panel  found  the  data  compelling  because  they  suggest  a  usable  metric  framework  that 
could  be  adopted  by  the  Marine  Corps. 
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NRftf  USMC  Immersive  Infantry  Efforts 


Infantry  Immersive  Trainer  (NT) 

-  Testbed  for  training  rifle  squads  in  current  theater  tactics 
and  decision-making 

-  Initial  implementation  at  Camp  Pendleton  (I  MEF) 

•  Simulates  a  small  Iraqi  village 

•  Portrays  realistic  engagements  with  indigenous  populations  (role 
players),  to  include  sights/sounds/smells 

-  Expanding  to  entire  Marine  Corps 

•  I  MEF  expansion;  II  MEF  facility;  III  MEF  implementation  in  MOUT  facility 

Squad  Immersive  Training  Environment  (SITE) 

-  Planned  as  a  POM12  POR  to  provide  a  truly  immersive  training 
environment  enabling  squads  to  train  across  a  full  range  of  missions. 

-  Capabilities-Based  Assessment  (CBA)  ongoing  to  identify  gaps 


The  Panel  made  a  site  visit  to  the  I  MEF  Infantry  Immersive  Trainer  at  Camp 
Pendleton,  CA.  The  IIT  is  the  Marine  Corp’s  flagship  for  immersive  training  for  small 
units  today  -  training  numerous  Marines  before  they  deploy  to  Iraq.  It’s  the  “test  bed”  in 
which  Marine  rifle  squads  and  other  small  units  receive  pre-deployment  and  refresher 
training  in  an  environment  that  resembles  an  Iraqi  village.  There  are  role  players  in  Iraqi 
wardrobe  -  encountered  in  their  homes,  mosques,  and  shops  along  with  computer 
generated  two-dimensional  human  avatars  projected  on  walls,  transcreens  or  displayed  on 
flat  panel  monitors.  There  are  simulated  explosives  -  many  of  the  elements  of  Southwest 
Asian  Counter  Insurgency  operational  venue  permeated  with  realistic  visual,  auditory, 
and  olfactory  stimuli. 

The  IIT  exposes  the  young  Marine  to  the  environment  of  indigenous  sights, 
sounds  and  smells  to  practice  small  unit  tactics  and  his  decision  making  skills  including 
“cueing”  and  “sense-making”.  Based  on  our  own  personal  immersive  demonstration  at 
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the  IIT,  followed  by  discussions  with  the  trainers  and  trainees,  the  Panel  is  confident  that 
the  training  is  vivid  and  realistic. 

The  current  I  MEF  IIT  will  be  expanded,  and  IIT-like  immersive  trainers  are 
planned  for  II  MEF  at  Camp  Lejeune,  NC;  and  for  III  MEF  in  Oahu,  HI,  in  the  next  year. 
A  2012  Program  of  Record  (POR)  will  develop  an  overarching  immersive  simulation 
toolset  for  the  Marine  Corps  dubbed  the  Squad  Immersive  Training  Environment  (SITE). 
Lessons  learned  from  the  IIT  training  as  well  as  from  other  developments  will  inform  the 
development  of  SITE. 
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NT  Observations 


•  Immersive  human-in-the-loop  live  environments 
like  the  IIT  will  remain  a  scarce  resource 

•  To  maximize  benefit  of  IIT,  users  could  utilize 
inexpensive  (e.g.  desktop)  pre-training  resources 

•  Decomposition  of  training  could  reduce 
dependence  on  live-environments  such  as  IIT 


In  April  2009,  the  Panel  spent  the  better  part  of  a  day  at  the  I  MEF  Infantry 
Immersive  Trainer.  The  agenda  included  briefings  by  the  trainers,  a  walk-through  with 
example  immersive  effects,  and  an  open  discussion  with  selected  IIT  “graduates”  from  all 
ranks. 


The  IIT  was  converted  from  a  former  tomato-packaging  plant  into  a  32,000- 
square-foot  military  urban  training  center  at  Camp  Pendleton,  California.  It  uses  sets 
designed  to  simulate  a  Southwest  Asian  village  (in  sight  and  smell),  an  ambient  sound 
system  and  special  effects.  Transcreen  hologram-type  displays  and  pyrotechnics  (IEDs 
and  RPGs)  lend  combat  realism.  Marine  Rifle  Squads  are  equipped  with  paint-ball  style 
weapons.  Marines  walk  through  realistic  dwellings,  alleys  and  other  settings, 
encountering  civilians  and  enemy  combatants. 

For  the  foreseeable  future,  immersive,  human-in-the-loop  live  environments  like 
the  IIT  will  remain  a  scarce  resource  due  to  the  costs  of  live  role  players  and  the 
infrastructure  investment  required  to  design,  implement  and  maintain  the  evocative 
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settings  for  exercises.  Based  on  the  comments  of  USMC  users,  the  IIT  appears  to  offer 
valuable  training  experiences. 

Concluding  the  IIT  visit,  the  Panel  agreed  with  the  observations  of  users  that  the 
greatest  value  of  these  training  exercises  is  the  trainees’  interaction  with  the  live  role- 
players  who  act  as  Iraqi  nationals,  both  as  friendly  civilians  and  insurgents.  The  panel 
believes  that  live  role-players  will  remain  at  the  IIT  and  similar  trainers  for  the 
foreseeable  future.  Simulated  humans  at  this  level  of  cognition  displayed  in  a  compelling 
fashion  are  not  in  anyone’s  technology  forecast,  even  when  considering  potential 
progress  over  multiple  decades. 

As  a  limited  resource,  the  IIT’s  impact  could,  however,  be  augmented.  The  Panel 
sees  a  need  to  leverage  what  the  IIT  offers  by  more  targeted  pre-training  in  advance  of 
Marines  coming  to  the  IIT.  Today,  all  Marines  receive  training  in  a  variety  of  individual 
skills  that  are  required  for  a  successful  IIT  evolution.  What  may  be  missing  in  pre¬ 
training  for  the  IIT  is  in  the  area  of  higher-order,  cooperative,  decision-making  exercises. 
This  pre-training  could  involve  squad-level  desktop  computer  simulation.  The  Panel  has 
no  specific  recommendations,  but  this  should  be  reviewed. 

Additionally,  the  desired  training  effects  -  and  corresponding  cognitive  skill 
capability  required  for  exercises  at  the  IIT  evolution  -  could  be  “teased  apart”  to 
determine  other  means  of  accomplishing  them.  The  next  chart  shows  examples  of  this. 
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NRRE Training  Decomposition:  Example 


TASK 

EXAMPLE  VENUE 

Cultural  "Norming" 

Desktop 

Cultural  "Taboo" 

Desktop 

"Freezing"  /  PTSD 

Virtual  Reality 

Decisions  Under  Stress  Mi: 

xed  or  Augmented  Reality 

Crowd  Control 

Mixed  Reality 

In  this  chart,  broad  categories  of  skills  required  for  Irregular  Warfare  are  listed 
with  suggested  possible  cognitive  training  venues.  In  this  case,  all  but  Crowd  Control 
could  be  experienced,  the  panel  believes,  in  a  meaningful  way  prior  to  exercise 
participation  at  the  IIT. 

Thus,  while  all  of  the  items  listed  in  this  chart  could  be  addressed  in  an 
institutional  immersive  environment  like  the  IIT,  the  panel  believes  that  students  could 
maximize  the  benefit  of  their  time  in  such  exercises  with  prior  experience  in  other,  more 
modest,  training  venues.  This  could  reduce  USMC  reliance  on  a  scarce  resource  like  the 
IIT. 
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ONR  Codes  30/341 
SITE  Enablers  ($K1 


Total:  $33,676,000  Over  8  Years 


Effort 

FY08 

FY09 

FY10 

FYll 

FY12 

FY13 

FY14 

FY15 

Expressive  Interaction  for 

Infantry  Simulation 

$400 

$440 

EC:  Naval  Next-generation 
Immersive  Technology  (N2IT) 

$2,400 

$3,950 

$4,900 

$4,500 

$3,100 

STTR:  Development  of  Low-Cost 
Tracking  System  for  Infantry 
Training 

$280 

$500 

$500 

STTR:  Development  of  Low-Cost 
Augmented  Reality  HMD 

$280 

$500 

$500 

Virtual  Environment  Prototyping 

$215 

$775 

$700 

$850 

$850 

$900 

$925 

Workload,  Stress,  and 
Performance  in  Immersive 
Training 

$130 

$480 

$650 

Tools  for  Games-Based  Training  & 

Assessment  of  Human  $1,000  $260  $2,000  $1,451 

Performance 

Predictive  Modeling  of  3D-Cued 
Audition  in  a  Complex  Naval  Task 

$110 

$110 

SITE  Support:  $1,325  $1,855  $4,580  $6,791  $4,800  $5,800  $5,425  $3,100 

What  are  the  technology  investments  to  support  the  required  capabilities  for  the 
next  generation  of  virtual  simulations?  To  begin  to  answer  this,  the  Panel  reviewed  the 
Office  of  Naval  Research,  Code  30  (Expeditionary  Maneuver  Warfare  &  Combating 
Terrorism  Department)  investment  portfolio  in  the  Human  Performance  Training  & 
Education  technology  investment  areas  -  primarily  for  support  to  the  Marine  Corps. 
Code  30  established  a  Science  and  Technology  Objective  (STO)  that  is  specifically 
designed  to  support  the  previously-described  SITE  program,  planned  for  2012. 

There  is  a  broad  range  of  S&T  activity,  however  the  level  of  funding  for 
individual  projects  is  quite  small.  For  example,  while  there  is  some  work  to  support  the 
development  of  an  augmented  head-mounted  display,  the  investment  level  could  be 
categorized  as  “probing  and  discovery”  rather  than  what  would  be  sufficient  to  address 
the  requirement. 
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The  largest  investment  described  is  for  a  Future  Naval  Capability  (FNC)  that  is 
focused  on  connectivity.  It  involves  the  ability  for  different  simulation  components  to 
connect  and  also  to  collect  large  amounts  of  data  for  simulation  runs.  These  are  key 
requirements  but  will  not  address  some  the  interface  issues  or  the  position  location 
information  challenges  that  are  also  essential  for  immersive  simulation. 
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Specific  Marine  Corps  S&T 
Top  Level  View 

•  ONR  Submitted  PRESBUD  FY10  ~$1.8B 

-  Marine  Corps  is  ~$146M  (8.1%  of  ONR  budget)* 

-  ONR  Code  30:  ~$110M;  MCWL:  ~$36M 

•  Panel  Observation  1 

-  Significant  percentage  increase  in  Marine  Corps  support  from  ONR 
budget  allocated  to  S&T  underpinning  of  Expeditionary  Maneuver 
Warfare  Applications  (approx.  46%  increase  over  past  four  years) 

•  Panel  Observation  2 

-  Establishment  of  Code  30  at  ONR  affords  Corps  opportunity  for  strategic 
leverage  and  focus 

-  For  the  SITE  initiative,  Code  30  investments  covering  many  areas  (  eight) 
with  too  little  funding  (~$5M/year)-  unlikely  to  produce  leap-ahead 
capability  or  achieve  significant  leverage 

*Note:  Does  not  include  $17M  Joint  Non-Lethal  Weapons  S&T 


In  2005,  the  decision  by  ONR  and  the  Marine  Corps  to  establish  Code  30  was  an 
opportunity  for  strategic  focus  and  leverage.  Currently,  about  8%  of  the  total  ONR  FY10 
Presidential  Budget  Request  is  allocated  for  Marine  Corps  S&T,  approximately  $146M. 
This  amount  is  divided  between  the  Expeditionary  Maneuver  Warfare  and  Combating 
Terrorism  S&T  Department  (Code  30),  and  Marine  Corps  Warfighting  Laboratory 
(MCWL).  It  represents  a  46%  increase  over  the  funding  of  PBR2006.  It  is  a  result  of  the 
recognition  by  the  Marine  Corps  and  the  Chief  of  Naval  Research  (CNR)  to  address  the 
inadequate  level  of  funding  for  developing  Expeditionary  Maneuver  Warfare  capabilities. 
In  view  of  the  importance  that  the  Secretary  of  Defense  has  recently  placed  on  the 
requirements  of  our  ground  forces,  the  Marine  Corps  and  the  CNR  will  need  to  increase 
their  focus  on  these  important  requirements. 

.  In  the  Panel’s  judgment,  the  current  array  of  SITE  programs  are  mostly  sub- 
critical  investments  and  are  unlikely  to  produce  leap-ahead  capabilities  or  enable  other 
efforts  in  the  Services  or  DARPA  to  be  leveraged  effectively.  Code  30,  with  Marine 
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Corps  support,  should  begin  a  prioritization  of  SITE  products  in  order  to  focus  on  fewer, 
more  significant  higher-priority  investments. 
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Other  Current  DoD 
S&T  IS  Investments 

•  FITE  JCTD:  An  effort  to  integrate  current 
capabilities  to  develop  an  overarching 
operational  utility  assessment.  Emphasis 
is  on  scenario-based  training. 

•  ICT:  Army-backed  university  research 
center  focused  on  the  artificial 
intelligence  aspect  of  the  immersive 
simulation  challenge 

•  RealWorld:  DARPA  program  developing  a 
simulation  software  application 
authorable  by  non-technical  users 

•  ULTRA-Vis:  DARPA  program...  While  not  focused  on  training  per  se,  this  effort 
to  create  a  lightweight  augmented  reality  display  and  gesture-based  control 
system  may  have  application  in  the  training  community;  Funded  in  Phase  I; 
Phase  II?  FY09-11 


2007  2008  2009  2010  2011  2012  2013 


■  ICT  ■  Real  World 

■  FITE  ■  ULTRA-Vis  Ph  I  (2  Teams) 

■  ULTRA-Vis  Ph  II  (1  Team)  ■  ULTRA-Vis  Ph  III  (1  Team) 


Beyond  the  ONR  activities,  the  Panel  also  looked  at  other  Department  of  Defense 
investments  in  this  S&T  area.  While  not  directly  related  to  the  focus  of  the  study,  these 
developing  programs  may  hold  potential  for  advancing  the  required  immersive  simulation 
training  technologies.  The  Joint  Forces  Command  is  sponsoring  the  Future  Immersive 
Training  Environment  (FITE)  Joint  Capabilities  Technology  Demonstration  (JCTD).  As 
a  technology  demonstration,  the  FITE  JCTD  will  determine  what  is  currently  possible, 
available,  and  able  to  be  integrated  into  other  training.  It  will  provide  a  comprehensive 
market  survey  of  the  technology  space  -  but  will  not  actually  advance  the  technology. 

The  Institute  for  Creative  Technologies  (ICT)  was  created  by  the  Army  in  1999  to 
revolutionize  the  learning  experience  through  active  engagement  with  the  Entertainment 
Industry  -  attempting  to  leverage  their  significant  investments.  This  would  lead  to  the 
development  of  interactive  digital  media  with  a  focus  on  advancing  artificial  intelligence 
(AI)  for  computer-generated  virtual  humans,  photo-realistic  computer  graphics  and 
learning  technologies  generally.  The  Army  investment  in  the  ICT  is  projected  to 
continue,  but  most  subject  matter  experts  doubt  that  the  actual  transition  of 
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comprehensive  AI  for  avatars  will  occur  for  “decades”.  Some  experts  will  say  “not  in  my 
lifetime”. 


Real  World  was  a  relatively  short-term  activity  at  DARPA  that  produced  a  set  of 
applications  with  a  single  goal  -  to  make  it  possible  for  non-technical  users  to  build  their 
own  simulations.  This  is  important  because,  if  an  infantry  commander  wants  to  create  a 
bespoke  simulation  (i.e.  one  custom-made  to  his  own  specifications),  with  a  terrain  mesh 
and  activities  to  challenge  his  unit  in  particular,  it’s  an  enabler.  But,  it  doesn’t  really 
solve  those  hard  immersive  simulation  challenges  that  the  panel  determined  were 
significant  obstacles  to  Marine  Corps  small  unit  immersive  simulation  training. 

DARPA’s  Urban  Leader  Tactical  Response,  Awareness  Visualization  project 
(ULTRA-Vis)  is  a  tactical  (vice  training)  system  that  is  currently  approaching  Phase  1 
completion.  One  of  its  key  components  is  a  lightweight,  head-mounted  display.  While  it 
may  not  have  the  kind  of  acuity  and  optical  characteristics  that  are  necessary  to  produce  a 
high  quality,  immersive  training  experience,  it  will  likely  advance  the  state  of  the  art  in 
head-mounted  displays.  However,  follow-on  work  will  be  necessary  -  in  a  Phase  2  and 
possibly  Phase  3  -  to  advance  the  technology  for  transition  to  a  practical  tactical  system. 
Further  work  will  then  be  required  to  develop  an  ULTRA-Vis  based  advanced  immersive 
training  display  system. 
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Conclusions 
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Current  Immersive 
Simulation  Limitations 


*  Cost/flexibility  of  fixed  infrastructure  investment 
(layout,  buildings,  scenic) 

*  Cost  of  role  players 

*  Absence  of  systematic  measurements 

*  After  Action  Review  (AAR) 


During  the  course  of  this  study,  the  panel  concluded  that  there  are  certain 
limitations  in  the  current  tools  available  for  immersive  simulation.  As  an  example,  when 
considering  the  Infantry  Immersive  Trainer,  it  will  be  noted  that  it  is  currently  configured 
to  simulate  an  Iraqi  village.  This  involves  environment-specific  scenery,  buildings, 
landscaping,  and  equipment.  It  stands  to  reason  that  making  substantial  changes  to 
represent  another  geo-specific  area  (e.g.  Afghanistan)  will  involve  significant  effort  and 
cost.  Considering  the  uncertain  and  continuously  changing  nature  of  future  Marine  Corps 
operations,  it  would  be  desirable  to  have  more  flexibility  in  configuring  a  structure  for 
immersive  simulation  training. 

In  addition  to  the  physical  set  utilized  at  the  IIT,  the  extensive  use  of  role  players 
is  a  limiting  factor.  Participants  who  had  experienced  training  at  the  IIT  were  unanimous 
in  their  agreement  that  the  role  players  were  the  most  important  component  of  their 
rotation  at  the  facility.  The  cost  of  role  players  is  high,  there  are  limitations  on  role 
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player  availability,  and  the  hours  they  can  work.  Recruiting  and  training  more  role 
players  is  potentially  a  time-consuming  and  expensive  proposition. 

To  this  point,  there  has  been  no  systematic  collection  of  data  at  the  IIT.  More 
extensive  instrumentation  is  needed  to  provide  baseline  data  for  comparative  evaluations 
of  training  goals  and  success.  Additional  metrics  may  need  to  be  considered  as  well, 
perhaps  along  the  lines  that  Dr.  Roman  suggested  in  his  Canadian  study. 

The  After  Action  Review  is  somewhat  limited  at  the  IIT.  Basically,  the  AAR  is  a 
playback  of  network  of  cameras’  video  streams,  like  a  civilian  security  surveillance 
console..  Although  reviewing  the  videos  provides  some  post-exercise  evaluation 
opportunity,  the  process  suffers  from  the  absence  of  metrics. 
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NRRC  Mitigating  Current  Limitations 

•  Cost/flexibility  of  fixed  infrastructure  investment 

-  Need:  Practical  Augmented  Reality 

-  Enabling  Technologies:  Practical  Head-Mounts,  Position/Location 
Information 

•  Cost  of  role  players 

-  Need:  Compelling  Virtual  Characters 

-  Enabling  Technologies: 

•  Near  Term:  No  solution 

•  Medium  Term:  Supervisory  control 

•  Long  Term:  Artificial  intelligence  research 

•  Absence  of  systematic  measurements 

-  Need:  Measurement  Protocol 

-  Enabling  Technology:  Data  capture  and  analysis 

•  After  Action  Review  (AAR) 

-  Need:  Three  Dimensional  Navigable  AAR 

-  Enabling  Technology:  Position/Location  Information 


There  are  a  number  of  possibilities  for  mitigating  some  of  the  limitations  of  the 
current  IIT  configuration.  First,  in  order  to  avoid  the  high  costs  of  Marine  Corps’ 
Military  Construction  (MILCON)  for  additional  fixed  infrastructure  environments,  better 
use  of  practical  augmented  reality  is  needed.  For  example,  using  head-mounted  displays 
could  create  a  system  in  which  new  environments  could  be  projected  for  the  user  in  the 
normal  field  of  view.  This  approach,  however,  will  require  the  development  of  practical 
and  affordable  lightweight  optical  see-through  HMDs  along  with  MOUT  site-sized, 
exterior-capable  high-resolution  Position/Location  Information  (PLI)  technology. 

To  overcome  the  high  cost  of  the  role  players,  there  is  a  striking  need  for 
compelling,  interactive  virtual  characters.  While  some  research  is  underway  in  this  area, 
there  is  not  a  near-term  solution  evident  and  may,  in  fact,  be  decades  away  or  longer.  In 
the  mid-term,  however,  use  of  supervisory  control,  as  currently  expressed  in  the  robotics 
domain,  could  possibly  enhance  the  use  of  virtual  characters.  In  this  scheme,  a  single 
human  operator  would  control  a  number  of  virtual  characters/avatars  (specific  limitations 
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to  be  determined),  and  would  develop  interactions  among  those  characters  as  well  as  with 
other  characters  controlled  by  other  humans,  as  well  as  with  the  participants  in  the 
immersive  simulation  training.  As  stated  previously,  absolute  use  of  artificial 
intelligence  to  create  life-like  human  avatars  is  extremely  challenging  and  probably  not 
possible  for  many  decades. 

In  the  opinion  of  the  Panel,  the  most  straight-forward  improvement  to  the  current 
IIT  is  to  add  systematic  evaluation  measurements  or  metrics.  Having  systematic  metrics 
will  allow  commanders  to  better  ascertain  unit  readiness  and  the  relative  value  of  a 
specific  immersive  training  evolution.  It  does  not  eliminate  the  subjective  evaluation  of 
subject  matter  experts;  rather  it  captures  their  observations  and  opinions  in  a  systematic 
fashion  that  can  be  incorporated  into  a  more  rigorous  evaluation.  An  example  previously 
discussed  was  the  work  of  Dr.  Roman  in  assessing  tank  commander  training.  He  utilized 
expert  evaluators  to  compare  student  performance  in  two  groupings  -  those  with  some 
exposure  to  simulation  training  and  those  who  only  had  didactic  training  in  a  classroom. 
Even  this  admittedly  basic  systematic  measurement  scheme  can  provide  the  basis  for 
comparisons  of  different  immersive  simulation  environments  -  an  eventual  step  forward 
for  acquisition  trade-offs. 

Finally,  substantial  improvements  can  be  made  to  the  After  Action  Review  with 
the  employment  of  a  three-dimensional,  high  fidelity  navigable  system.  This  would 
incorporate  the  objective  PLI  capability  previously  mentioned,  as  well  as  implementing 
the  systematic  capture  of  the  data. 
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NRRC  Navigable  AAR 


Vision  cones:  instantaneous  field  of  view 
of  USMC  Fire  Team  viewed  from  above 


Two  examples  of  full  3-D  AAR  screen-shots  are  shown  above.  An  instrumented, 
three-dimensional  navigable  system  provides  high  data  fidelity  for  accurate  positioning 
and  location  of  each  participant. 

Threat  lines  are  displayed  showing  an  individual  Marine’s  vulnerability  to 
OPFOR  targeting  and,  conversely,  his  opportunities  to  return  fire.  A  cone-of-vision  view 
can  highlight  missed  tactical  surveillance  coverage.  This  capability  enables  a 
comprehensive  post-training  assessment  of  the  instantaneous  threat  conditions 
encountered  by  the  squad  or  fire  team  during  the  session. 
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Recommendations 
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NRRC  Immediate  Implementation 


•  Create  systematic  measurement 

-  Subjective  assessment  by  a  trained  evaluator  is  a  valid  metric 

•  Develop  T&R  Manual  METs  to  include  cognitive  aspects  of 
irregular  warfare  training  requirements 

-  Approve  developed  METs  for  the  NT  and  Next-Gen  MOUT 
•  Map  deconstructed  METs  to  available  alternatives 

•  NT  improvements:  "low-hanging  fruit" 

-  Employ  alternative  face  protection  for  improved  transparency 

-  Enhanced  "4D"  cues 

-  E.g.  Sound  reinforcement/propane  effects  to  emphasize  explosives/energetics  events. 

•  Cognitive  task  analysis 

-  Decompose  training  objectives  for  "end-to-end"  solution 

•  Implement  small  unit  system  prototype  laboratory  facility 

-  Create  experimentation,  test  and  evaluation  schedule  at  Camp  Pendleton 
I  IT  (requires  indoor/outdoor  facility) 


Any  initiative  to  implement  immersive  simulation  training  across  the  Marine 
Corps  will  not  likely  survive  without  strong,  vocal  support  from  the  top  leadership. 
Additionally,  the  initial  transition  from  S&T  and/or  the  operational  communities  to  an 
acquisition  program  is  likely  to  be  high-risk  unless  specific  and  measurable  requirements 
are  first  established  -  along  with  the  appropriate  metrics. 


The  Panel  recommends  the  following:  first,  create  a  systematic  measurement 
program  for  existing  immersive  training  simulations;  second,  adjust  the  mission-essential 
tasks  (METs)  in  the  T&R  Manual  to  include  the  cognitive  decision-making  aspects  of 
irregular  warfare.  Then  begin  mapping  the  new,  evolving  METs  into  the  available 
simulation  training  alternatives.  These  alternative  training  approaches  can  then  be 
validated  using  the  systematic  measurement  system. 


Third,  “Low-hanging  fruit”  opportunities  exist  today  that  can  improve  the 
effectiveness  of  the  ITT.  A  fairly  intuitive  example  is  sound  reinforcement  to  enhance 
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realism  -  not  unlike  the  high- volume,  surround-sound  of  a  Hollywood  movie.  At  present, 
when  a  simulated  explosion  occurs  in  the  IIT  -  it’s  loud  -  but  it  lacks  the  full  spectrum 
acoustical  force  that  accompanies  a  real  explosive  event.  By  building  up  the  IIT  sound 
capability,  the  Marines  would  be  offered  the  full  inffasound  (low  frequency  sound 
pressure)  experience  to  gain  a  higher  degree  of  immersive  effect. 


Fourth,  attempt  decomposing  the  basic  training  objectives  -  especially  in  the 
higher-order  cognitive  tasks  -  to  discover  which  sub-tasks  could  be  enhanced  through 
simulation  training.  Initial  studies  done  at  the  University  of  Central  Florida,  under 
contract  to  ONR,  show  that  some  task  learning  is  better  achieved  using  simulation 
training  than  in  the  real  world. 


Finally,  the  Panel  strongly  recommends  that  the  ITT  be  turned  into  a  laboratory 
after  hours.  Operations  currently  end  when  it’s  quitting  time  -  and  everyone  goes  home. 
The  ITT  could  easily  be  used  during  off-training  hours  to  develop  and  test  new  training 
simulations,  new  training  scenarios,  and  additional  training  effectiveness  measures.  This 
“laboratory  at  night”  would  require  adequate  resourcing  for  planning,  staffing,  and 
technology  insertion. 
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SITE  S&T  Way  Ahead: 

A  Game-Changer  in  Training  Capability 

•  Establish  a  3-5  year  program  at  the  Camp  Pendleton  NT  small  unit-level 
system  laboratory  to  provide  capability  and  system  integration  to  advance 
next-generation  immersive  simulation  capability-  practical  Augmented 
Reality 

-  Augmented  reality  display:  an  HMD  with  high  resolution/fidelity,  zero  latency,  and  minimal 
"overhead." 

-  Position/Location  Information  capability:  High  accuracy  tracking  of  all  entities. 

-  Compelling  virtual  characters  with  supervisory  control 

-  High-fidelity  After  Action  Reviews 

*  Review  and  assess  alignment  of  currently  funded  ONR  Code  30/34  Enabling 
Capabilities  to  support  SITE  needs 


As  stated  earlier,  an  immersive  simulation  training  capability  will  probably  not 
survive  the  Pentagon  “budget  battles”  without  strong  and  consistent  senior  Marine  Corps 
leadership  support. 


That  said,  the  NRAC  Panel  believes  that  the  initiation  of  a  significant  immersive 
simulation  training  program  for  transition  to  the  operating  forces  will  be  perceived  as 
high  risk  to  the  acquisition  community.  Therefore,  the  Panel  strongly  recommends  that 
the  Marine  Corps  create  a  small  unit  training  system  laboratory  or  test  bed  -  probably 
using  the  existing  IIT  facility  during  its  daily  downtime  -  to  allow  the  early  introduction 
of  emergent,  immersive  simulation  technologies  to  be  integrated  in  an  orderly  systems 
approach.  The  ONR  SITE  Program  will  be  investing  in  a  number  of  requisite  immersive 
technologies.  The  Army  and  DARPA  also  may  have  technologies  of  interest.  But,  even  if 
individual  immersive  components  are  successfully  developed  -  the  community  won’t 
know  -  the  Marine  Corps  won’t  know  -  if,  in  fact,  they  will  work  together  to  create  a 
satisfactory  product. 
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Finally,  the  Panel  encourages  a  detailed  review  of  all  ONR  Code  30/34  enabling 
technologies  to  ensure  the  proper  priorities  are  assigned  to  support  SITE.  The  goal  is  to 
create  practical  augmented  reality  to  provide  a  game-changing  capability  for  Marine 
Corps  training. 
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NRRC  Immersive  Simulation  Road  Map 


NT  Testbeds 
ONR  Enabling  S&T 

FITE  JCTD 

DARPA  ULTRA-vis 


I  MEF  IIT  Experimentation,  T&E 


I  MEF 


II,  III  MEF  plus  Upgrades 


Mature  Technologies  to  TRL  6  (SITE  Related  STOs) 


Demo  TRL  6+  Technologies 


Phase  I  Phase  II  -  III 


This  chart  shows  a  macro  level  road  map  for  advancing  infantry  immersive 
simulation  in  the  Marine  Corps.  It  identifies  the  key  science  and  technology  enablers  and 
activities  currently  underway  in  this  technology  space.  It  is  not  known  if  the  DARPA 
ULTRA-Vis  will  be  funded  beyond  Phase  I,  but  the  program  merits  support  and  needs  to 
be  monitored.  The  FITE  JCTD  will  demonstrate  Technology  Readiness  Level  (TLR)  6 
and  above  technologies  that  may  be  appropriate  for  transition  into  Squad  Immersive 
Training  Environment  (SITE). 

The  ONR  Code  30  and  34  enabling  technologies  relating  to  Human  Performance 
Training  &  Education  should  continue  to  be  funded  and  advanced.  The  Marine  Corps 
has  funding  for  and  will  stand  up  Infantry  Immersive  Trainer  simulation  centers  for  II 
MEF  and  III  MEF  during  2010. 

To  support  technology  experimentation,  and  test  &  evaluation,  the  Panel 
recommends  that  the  Marine  Corps  identify  one  of  the  IIT  test  bed’s  to  be  co-used  as  a 
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Small  Unit  Laboratory.  Because  the  I  MEF  IIT  will  have  the  capability  to  support  both 
indoor  and  outside  training,  we  recommend  the  Lab  be  co-located  at  the  current  Camp 
Pendleton  site. 
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NRRC  ONR  Long-Term  Research  Questions 


•  How  realistic  can  this  be  made: 

-  Automated  scenario  creator  and  controller 

-  Automated  role-players  (individuals  and  crowds) 

-  Ability  to  move  in  real  3D  space  while  in  simulation 

-  Quickly  inducing  physically  and  mentally  stressed  trainee 

-  Rapidly  reconfigurable  settings 

•  Need  to  understand: 

-  Role  of  stress  in  decision-making,  and  learning  & 
retention 

•  Need  to  measure  (lab  &  in  the  field) 

•  Need  to  understand  types  &  effects 

•  Ability  to  induce  &  modulate  high  stress 


The  research  community  has  undertaken  many  of  the  challenges  associated  with 
the  development  of  the  technologies  for  immersion  simulation  training.  Today,  there  are 
cockpit  trainers  that  are  so  immersive  -  for  both  pilot  training  and  evaluation  -  that  the 
Services  and  the  Federal  Aviation  Administration  (FAA)  allow  their  substitution  for 
much  of  the  actual  flying  syllabus.  Unfortunately,  this  level  of  maturity  has  not  been 
reached  for  immersive  small  unit  infantry  training  which  necessarily  includes  an  almost 
limitless  variety  of  localities,  environments,  and  threats. 

The  Panel  found  a  number  of  early  research  efforts  at  ONR  and  elsewhere  that  - 
if  fruitful  -  will  enable  simulations  to  become  more  realistic  to  enhance  the  learning 
process  and  to  decrease  the  operator  costs  associated  with  humans-in-the-loop.  As  shown 
above,  examples  of  these  involve  the  automation  of  critical  portions  of  the  training 
environment:  role-players,  the  training  scenario  itself;  as  well  as  individual  mobility  in  an 
immersive  environment.  These  are  not  easy  problems  but  the  investment  in  these  areas 
should  continue. 
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There  are  also  areas  that  need  more  emphasis.  These  are  longer-term  scientific 
questions  that  get  to  the  heart  of  military  immersive  training:  what  is  stress,  how  can  it  be 
measured,  how  can  it  be  safely  induced  in  a  training  situation  so  it  may  “inoculate”  the 
trainee  to  its  worst  physiological  and  psychological  effects. 
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Summing  Up 


•  ASNRDA 

-  Establish  a  Community  of  Interest  to  address  issues  related  to  Infantry 
Immersive  Simulation.  Participants  to  include  Navy,  Marines,  Army,  DD&RE, 
and  DARPA.  Expand  as  appropriate. 

•  DC,  CD&I 

-  Implement  systematic  evaluation  of  immersive  training  alternatives. 

-  Enhance  NT:  "Low-hanging  fruit" 

•  CNR 

-  Establish  the  Small  Unit  S&T  Laboratory  capability  for  the  proposed  SITE  POR 

-  Examine  Code  30/34  budget  priorities  for  immersive  simulation  training  needs 

-  Increase  long-term  research  for  immersive  training. 


This  figure  shows  the  Panel’s  top-level  recommendations,  with  appropriate 
assignments  for  action. 

Because  it  is  likely  that  the  Marine  Corps  will  continue  to  supply  ground  combat 
forces  to  various  theaters  for  the  foreseeable  future,  the  ASN  RDA  should  be  the 
Pentagon  advocate  for  the  establishment  of  an  immersive  training  “Community  of 
Interest”.  As  Lt  Gen  Flynn  has  stated,  “the  goal  must  be  to  take  training  capabilities  to 
the  next  level  and  fuse  current,  emerging,  and  future  live  and  virtual  technologies  to 
create  a  fully-immersive  live/virtual  training  environment”.  This  new  group  would 
address  issues  related  to  infantry  immersive  simulation  with  participation  from  the  Navy, 
Marines,  Army,  DD&RE,  and  DARPA. 

The  Deputy  Commandant  for  Combat  Development  &  Integration  (DC,  CD&I) 
should  implement  a  systematic  evaluation  of  immersive  training  alternatives  to  make 
near-term  capability  improvements.  This  evaluation  should  address  “low  hanging  fruit” 
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for  the  IIT  such  as  the  addition  of  enhanced  acoustics  and  sound  reinforcement  which  can 
greatly  improve  simulated  reality  effects. 

The  Panel’s  critical  recommendation  is  for  the  CNR  to  establish  the  Small  Unit 
S&T  Laboratory  to  explore  the  development  issues  for  a  SITE  capability.  Also,  the  CNR 
should  evaluate  budget  priorities  in  ONR  for  supporting  Code  30;  its  current  efforts  are 
spread  widely  and  seem  unlikely  to  produce  significant  results. 

If  all  of  these  recommendations  are  followed,  then  as  GEN  Mattis  (Commander, 
US  Joint  Forces  Command)  stated  “...a  giant  leap  forward  in  our  simulated  training 
environment  for  small  units  in  ground  combat . .  .to  replicate  to  the  degree  practical  using 
modem  simulation,  combat  scenarios  that  will  test  our  small  units  ...”  will  have  been 
addressed. 

The  objective  at  the  end  of  the  day  is  for  a  Marine  -  facing  live 
combat  for  the  first  time  -  will  have  already  experienced  numerous 
engagements  of  equivalent  intensity  but  without  equal  consequences .  So 
that  his  first  battles,  the  battles  that  harden  and  make  him  vastly  more 
likely  to  survive,  are  virtual. 
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Appendix  A 

Terms  of  Reference 

Terms  of  Reference 

Immersive  Simulation  for  Marine  Corps  Small  Unit  Training 
NRAC  Summer  Study  2009 

Objective 

Study  concepts  of  immersive  training  simulation  to  assist  Marines  in  developing  complex 
and  ‘intuitive’  decision  skills  while  under  the  attendant  conditions  of  physical  and 
emotional  stress.  This  study  will  examine  how  immersive  simulations  may  have  the 
potential  to  reduce  the  pre-deployment  time  required  to  train  small  unit  leaders  over  that 
of  other  contemporary  training  approaches  and  methods.  The  study  should  expand  on 
current  concepts  and  provide  insights  on  new  Marine  Corps  training  for  the  small  unit. 
Background 

The  complexity  of  the  Contemporary  Operational  Environment  and  the  enduring  need  for 
combat  readiness  across  the  full  spectrum  of  conflict  and  environmental  extremes  place 
new  demands  on  the  decision-making  skills  of  Marine  Corps  small  unit  leaders.  Making 
the  best  tactical,  legal  and  ethical  decisions  under  conditions  of  emotional,  psychological, 
and  physical  duress  has  stimulated  great  interest  in  new  kinds  of  training  and  simulation 
to  prepare  these  leaders  for  combat.  With  a  goal  of  developing  ‘experienced’  decision¬ 
making  ability  prior  to  actual  deployment,  new  forms  of  immersive  training  simulation 
may  assist  Marines  in  developing  these  complex  and  ‘intuitive’  decision  skills  while 
under  the  attendant  conditions  of  physical  and  emotional  stress.  Moreover,  immersive 
simulations  may  have  the  potential  to  reduce  the  pre-deployment  time  required  to  train 
small  unit  leaders  over  that  of  other  contemporary  training  approaches  and  methods. 
Simulating  the  most  critical  of  combat  conditions  is  both  necessary  and  extremely 
challenging. 

Specific  Tasking 

•  Decompose  this  problem  and  identify  the  desired  effects  that  must  be  produced  by 
such  training.  With  the  problem  deconstructed  and  fully  defined,  examine  the 
attendant  the  metrics  necessary  to  gauge  solution  (training)  effectiveness. 

•  Review  current  and  developing  virtual  training  methods  and  technologies  in  the 
DOD,  as  well  as  other  agencies  and  allies,  both  within  and  outside  of  the 
government. 

•  Identify  and  evaluate  S&T  initiatives  (US  and  allies)  that  are  being  pursued  and 
explore  opportunities  that  should  be  accelerated. 

•  Recommend  technology  solutions,  investments  and  developments  required. 
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Appendix  B 


Briefer  List 


Briefers 

Office/Organization 

Col  Frank  Kelley,  USMC;  Ms.  Nancy 

Harmon 

Marine  Corps  Program  Manager  for  Training  Systems 

Mr.  Jack  Sparks 

Marine  Corps  Warfighting  Lab,  Modeling  and 

Simulation  Branch 

Mr.  Dennis  Thompson,  Major  Brent 

Goodrum,  USMC,  Mr.  Earnest  King 

USMC  Training  and  Education  Command 

Mr.  Gary  Fraas 

Naval  Air  Warfare  Training  Center  Training  Systems 

Div,  Advanced  Simulation,  Visual  &  Software  Systems 

Division 

Mr.  Tim  Sayers,  Mr.  Don  Whitley,  Mr.  Rob 

Miller 

Army  PEO  Simulation,  Training,  and  Instrumentation 

Dr.  Charles  Hughes,  Professor 

University  of  Central  Florida  School  of  Electrical 

Engineering  &  Computer  Science,  Media  Convergence 

Laboratory 

Mr.  Jay  Reist,  Mr.  Clarke  Lethin,  Mr.  Pete 

Muller 

Future  Immersive  Training  Environment  (FITE)  JCTD 

Col  Chris  Macedonia,  USA,  M.D. 

Medical  Science  Advisor  to  the  Chairman  of  the  Joint 

Chiefs  of  Staff 

Mr.  George  Solhan,  Dr.  Roy  Stripling,  Mr. 

Pete  Muller 

ONR  Code  30  (Expeditionary  Maneuver  Warfare  & 

Combating  Terrorism  Department) 

Dr.  Mike  Macedonia,  Mr.  Bart  Bartlett 

Fortera  Federal  Systems 

Dr.  Steve  Goldberg 

US  Army  Research  Institute 

Mr.  Kevin  Cippant 

L-3  Com,  Link  Simulation  and  Training 

Mr.  Dell  Lunceford 

Total  Immersion  Software,  Inc. 

Mr.  Mark  Bolas,  Mr.  Kim  LeMasters 

Institute  for  Creative  Technology,  University  of 

Southern  California 

65 


Briefers 

Office/Organization 

Mr.  Jim  Mowery,  Mr.  Tom  Wilson 

Training,  Doctrine,  and  Combat  Development,  US 

Army  Armor  Center 

Dr.  Cali  Fidopiastis 

Applied  Cognition  and  Training  in  Immersive  Virtual 

Environments  Lab,  Institute  for  Simulation  and 

Training,  University  of  Central  Florida 

LTC  Chris  Niederhauser,  USA 

Fires  Battle  Lab,  US  Army 

Mr.  Jeff  Singleton 

Deputy  Director  for  Research,  Office,  Deputy  Assistant 

Secretary  of  the  Army  (R&T)  SAAL-TR 

Dr.  Amy  Vanderbilt 

DARPA 

BGen  Murray,  USMC 

Marine  Corps  Warfighting  Lab 

Dr.  Mark  Livingston 

Naval  Research  Lab 

Dr.  Jeff  Wilkinson 

MYMIC,  LLC 

Col  Craig  Langhauser,  USA 

US  Army  Development,  Research  and  Engineering 

Command,  Simulation  and  Training  Technology 

Center 

Dr.  Jonathan  Gratch 

Institute  for  Creative  Technology,  University  of 

Southern  California 

Ms.  Julia  Kim 

Institute  for  Creative  Technology,  University  of 

Southern  California 

Mr.  Ryan  McAlinden 

Institute  for  Creative  Technology,  University  of 

Southern  California 

Dr.  Mark  Core 

Institute  for  Creative  Technology,  University  of 

Southern  California 

Mr.  Ken  Falke 

A-T  Solutions 

LtGen  Flynn,  USMC 

Deputy  Commandant  for  Combat  Development  and 

Integration 

Mr.  Jeff  Bergenthal,  Mr.  Richard  Boyd 

Lockheed  Martin,  Samoff  Labs 

Dr.  Sae  Schatz 

University  of  Central  Florida 
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Briefers 

Office/Organization 

Mr.  Darrell  Smith,  Mr.  Rob  Lechner 

Boeing,  Integrated  Defense  Systems 

Training  Systems  &  Services 

Dr.  Michael  Van  Lent 

Soar  Technologies 

Dr.  Paul  Roman 

Royal  Military  College  of  Canada 

Mr.  Warren  Katz 

MAK  Technologies,  Inc. 

Dr.  Elaine  Rayboum,  Mr.  Dan  Small 

Sandia  Labs 
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Appendix  C 


Acronyms 


AAR 

After  Action  Review 

AI 

Artificial  Intelligence 

ASN(RDA) 

Assistant  Secretary  of  the  Navy  for  Research,  Development, 

Acquisition 

CNR 

Chief  of  Naval  Research 

CCS 

Combat  Convoy  Simulator 

DARPA 

Defense  Advanced  Research  Projects  Agency 

DC,  CD&I 

Deputy  Commandant  for  Combat  Development  & 

Integration 

DVTE 

Deployable  Virtual  Training  Environment 

FAA 

Federal  Aviation  Administration 

FITE 

Future  Immersive  Training  Environment 

FNC 

Future  Naval  Capability 

ICT 

Institute  for  Creative  Technologies 

IED 

Improvised  Explosive  Device 

IIT 

Infantry  Immersive  Trainer 

ISMT 

Indoor  Simulated  Marksmanship  Trainer 

MCWL 

Marine  Corps  Warfighting  Laboratory 

MEF 

Marine  Expeditionary  Force 

MILCON 

Military  Construction 

MET 

Mission-Essential  Task 

OPFOR 

Opposing  Force 

PLI 

Position/Location  Information 

PTp 

Pre-Deployment  Training  Plan 

RPG 

Rocket-Propelled  Grenade 

SITE 

Squad  Immersive  Training  Environment 

STO 

Science  and  Technology  Objective 
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T&R 

Training  and  Readiness 

TOR 

Terms  of  Reference 

ULTRA-Vis 

Urban  Leader  Tactical  Response,  Awareness  Visualization 
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